A novel “sound visualization” process in virtual 3D space: the human auditory perception analysis by ecological psychology approach

In this paper we propose “sound visualization” which provides a learning process for listeners considered as perceptual systems to perceive “character-related” sounds in a virtual 3D acoustic space. “Character-related” sound represents the most appropriate sound event to be perceived (“read”) even by the visually impaired. The test data (characters) used in our experiments were segmented from degraded images concerning real license plate photos. Multilayer Perceptron nets and sound space processor were used to transform these characters into perceptible auditory textures (non-speech audio) called “character related” sounds. Each “character related” sound perception accuracy was assessed by ten listeners in several experiments subjected to perceptual training and testing sessions. The term “sound visualization” describes the perception process of “character-related” sounds through our auditory system and such a perception analysis by ecological psychology approach. The proposed “sound visualization” was useful for people who have difficulties acquiring perceptual skills to “read” “character-related” sounds. The results show that relatively simple and short periods of perceptual training are enough to enhance listening abilities to classify sounds. Also the results indicate that perception depends on psychoacoustic learning mechanisms rather than simple spatial localization of acoustic signals. Furthermore, the work encourages us to conclude that “sound visualization” is a viable and great alternative for the visually impaired that use computers as communication media.